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Abstract 

We study the Hamilton-Jacobi equation for undiscounted exit time con- 
trol problems with general nonnegative Lagrangians using the dynamic pro- 
gramming approach. We prove theorems characterizing the value function 
as the unique bounded-from-below viscosity solution of the Hamilton-Jacobi 
equation that is null on the target. The result applies to problems with the 
property that all trajectories satisfying a certain integral condition must stay 
in a bounded set. We allow problems for which the Lagrangian is not uni- 
formly bounded below by positive constants, in which the hypotheses of the 
known uniqueness results for Hamilton-Jacobi equations are not satisfied. 
We apply our theorems to eikonal equations from geometric optics, shape- 
from-shading equations from image processing, and variants of the Fuller 
Problem. 
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1 Introduction 



Viscosity solutions form the basis for much current work in control theory and 
optimization (cf. [lHiniinil2Hll2i)-Ina recent series of papers (cf. [TTllTHllTniOni 

1221) J we presented results characterizing the value function in optimal control as the 
unique viscosity solution of the corresponding Hamilton- Jacobi-Bellman equation 
(HJBE) that satisfies appropriate side conditions. These results apply to very 
general classes of exit time problems with unbounded dynamics and nonnegative 
Lagrangians, including H.J. Sussmann's Reflected Brachystochrone Problem (cf. 

and other problems with non-Lipschitz dynamics (cf. |18[I20| ). They also 
apply to the Fuller Problem and eikonal equations where the Lagrangians are not 
bounded below by positive constants and may even vanish outside the target for 
some values of the control (cf. ^| I19L I20L \'2'2\). In this note, we extend some 
results of |17[ [T^ on proper viscosity solutions of the HJBE by characterizing the 
exit time value function as the unique bounded- from-below viscosity solution of 
the corresponding HJBE that is null on the target. (Recall that properness of a 
function u; : ^ M is the condition that w{x) —>■ +00 as — > 00, which 
is a more stringent requirement than boundedness from below.) This refinement 
applies to a large class of deterministic exit time problems for which the Lagrangian 
is not uniformly bounded below by a positive constant and for which an extra 
affordability condition (namely, {Hq) below) is also satisfied. We apply this result 
to several physical problems studied in ^| 12^1 , including eikonal and shape-from- 
shading equations, as well as variants of the Fuller Problem that arc not tractable 
using the well-known results or using our earlier results. (For example, see |29j . 
which imposes the requirement, which is not needed below, that the light intensity 
/ for shape- from-shading satisfies I{x) < C < 1 for all x and some constant C; 
(30) . which considers solutions of eikonal and shape- from-shading equations on 
bounded sets; ^1 |2S| for uniqueness of bounded solutions of shape-from-shading 
equations; and |22[ I29j which impose asymptotics, given in Hll|l below, which will 
not in general be satisfied for the problems we consider here.) 

Value function characterizations of this kind have been studied by many au- 
thors for a variety of stochastic and deterministic optimal control problems and 
for dynamic games. The characterizations have been applied to the convergence of 
numerical schemes for approximating value functions and differential game values 
with error estimates, synthesis of optimal controls, singular perturbation prob- 
lems, asymptotics problems, H°°- control, and much more. See for example p^ llHj 
and the hundreds of references in these books. For surveys of numerical analysis 
applications of viscosity solutions, see and for uniqueness characterizations 

for the HJBE of discounted exit time problems, see jH). For uniqueness charac- 
terizations for general Hamilton- Jacobi equations that do not necessarily arise as 
Bellman equations, see P El ■ ^or an appropriate stronger solution concept 
for a subclass of problems, leading to a characterization of a maximal solution as a 
unique solution, see However, these earlier characterizations cannot in general 
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be applied to exit time problems whose Lagrangians are not uniformly bounded 
below by positive constants. In fact, one easily finds exit time problems for which 
the Lagrangian is not bounded below by a positive constant and for which the cor- 
responding HJBE has more than one bounded-from-below solution that vanishes 
on the target. Here is an example from '19' where this occurs: 

Example 1.1 Choose the dynamics and Lagrangian 

x{t) ^u{t) £ [-1,1], e{x,a) = L{x) {x + 2f (x -2fx^{x + lf{x -1)^, (1) 

respectively. Let vi and V2 denote the value functions for the exit time problem of 
bringing points to the targets Ti = {0} and T2 = {0, 2, —2}, respectively, using the 
data (^3) (cf. © below). Therefore, if we let A4 denote the set of all measurable 
functions u : [0, cxo) — » [—1, +1], then 

Vj{x) = inf < / L{(j){s)) ds : ti,j < 00, 0(0) = x,4> = u a.e. > for j = 1, 2 
ueM [Jq ) 

where ti,j — 'mi{t > : 4){t) G 7^} for j — 1, 2. One can easily check that vi and 
V2 are both viscosity solutions of the associated HJBE 

\\Dv{x)\\ ^ix + 2f {x - 2fx^{x + lf{x ~ if (2) 

on M \ T with the target T := Tl that vanish on T . One checks that with the 
target T Ti, the problem satisfies all hypotheses of the well-known theorems 
that characterize value functions of exit time control problems as unique viscosity 
solutions of the HJBE that are zero on T (cf. |31[7||^) except that the positive 
lower bound requirement on I is not satisfied. I 



Remark 1.2 One of the hypotheses we will make on the exit time problems in the 
rest of this paper is that the running costs of trajectories starting outside T and 
running for any positive time are always positive (cf. condition (Hs) below). This 
positivity hypothesis is not satisfied in the previous example, since the trajectory 
x{t) = — 1 ^ T gives J„ L{x{s))ds = for all t. On the other hand, all other 
hypotheses we make in 32]below do hold for Example ll.ll Therefore, under the set 
of assumptions in our setting, condition {H^) cannot be removed. I 

This note is organized as follows. In 12 we introduce the notation and hy- 
potheses in force throughout most of the sequel, including the definitions of the 
exit time HJBE, relaxed controls and viscosity solutions. In fj^l we state our main 
result, and we also explain how this result improves what was already known about 
viscosity solutions of the HJBE. Our results apply to exit time problems that vi- 
olate the usual positivity condition on the Lagrangian (namely, (|10|l below) and 
that are also not tractable by means of ^| El El QIIl 122]' This is followed in 
by statements of the main lemmas. In we prove our main result, and SjHl 
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gives physical applications, including cases that are not tractable using the known 
results or any of our earlier results. This is followed in 33 by variants of our main 
result for discontinuous viscosity solutions and local solutions. We conclude in ^ 
by showing how to use the methods of [21 to extend our results to cases where 
the control set is unbounded. 



2 Definitions and Hypotheses 

This note is concerned with problems of the form 

For each x £ M^, infimize / e"" {yx{s, P), P{s)) ds 

Jq 

over all f3 ^ A for which tx{l3) < oo, (3) 
where yx{-, P) is defined to be the solution of the initial value control problem 

m^ny{t),m) a.e. , y{Q) = x (4) 

for each x G and each (3 ^ A := { measurable functions [0, oo) A^} for 
a given fixed compact metric space A and possibly unbounded nonlinear control 
system /, and tx{l3) := inf{t > : yx{t,l3) £ T} for a given fixed set T C R^. 
(Depending on /, some choices of x could give tx{P) = +oo for all f3, in which 
case the infimum for Q is +cxd.) Here, A^ denotes the set of all Radon probability 
measures on A viewed as a subset of the dual of the set C{A) of all real-valued 
continuous functions on A, and A has the weak-* topology, so A is the set of relaxed 
controls from jJlEIESI- Notice that A includes all measurable a : [0, oo) A, 
which can be viewed as Dirac measure valued relaxed controls, and that A^ is 
compact. We also consider Q for cases where A C R*^ is closed but not bounded, 
in which case we set A^ = A and 

A := {measurable fmictions [0, oo) : S C A compact} 

{measurable functions [0, oo) — > A} (5) 

which of course reduces to the usual definition of A when A is compact. For 
compact S C A and measurable a„,Q!,TO : [0, oo) — > S^, we set h^{x,m) :— 
Jg h{x, a)dm{a) for x e R^ and h = f,£ for suitable / and £ specified below, and 
an ^ a weak-* means that for alH > and for all Lebesgue integrable functions 
B : [0,t] C{S), we have 

lim / f {B{s))ia)dian{s))ia)ds= f f {B{s)){a) d{ais)){a) ds (6) 
"^°°7o Js Jo Js 

Also, recall that STCT is the small-time controllability condition that 

r C int {TZ") for all e > 0, 
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where 

n' {x e : 3t e [0, e) & /? G s.t. y^{t, f3) e T}. 

Roughly speaking, STCT means points near T can be brought to T in small time. 
We remark for later reference that STCT is a property of the restriction of the 
vector fields /(•, a) to neighborhoods of T . In most of what follows, we assume the 
following standing hypotheses (but see [JSlfor analogs for cases where the control 
set A is not assumed to be compact): 

{Hi) A is a nonempty compact metric space. 
{H2) T C is closed and nonempty, STCT. 

{H3) f is continuous, and 3L > such that ||/(a;,a) — f{y,a)\\ < L\\x — y\\ 
'\fx,y eR^ ae A. 

{H4) e -.R^ X A [0,00) is continuous. 

(iJg) lite (0,cx)), /3 e A and a; G \ T, then J* r{y.^{s, (3), (3{s)) ds > 0. 

(Hq) If X G and (3 e A are such that hmsupg^^o^ ||y^(s,/3)|| — 00, then 
/o°°r(y,(s,/3),/3(s))ds = +cx). 

Remark 2.1 Assumptions (7?5)-(i?6) are expressed in terms of the trajectories, 
rather than the HJBE data. From the PDE point of view, it is desirable to be able 
to check all of our assumptions directly from the data = (/i, f2, ■ ■ ■ , In), , and 
T from the PDE, rather than assuming complete knowledge of the trajectories. One 
set of conditions on the data implying (iJs) is (i) there are constants K > Q and 
C > such that a) > K\xi\'-^ for all a G A and x = {xi,X2, ■ ■ ■ , xjy) G M^, and 
(ii) if y G K^'^ and (0, y) G \ T, then ^ {/i(0, y, m) : m G A''}. Conditions 
(i)-(ii) ensure that there is a positive cost assigned to staying outside T on each 
interval of positive length. These conditions will hold for example in the Fuller 
Problem discussed below (cf. H6.1(l . By using a generalized version of "Barbalat's 
lemma", (Hq) can also be checked from the HJBE data (cf. |2J|> §2)- 

Before discussing the motivation for these hypotheses, note that by the Fil- 
ippov Selection Theorem (cf. (SSl)i all of our results remain true if A is replaced 
by {measurable functions [0, 00) — > A} throughout the preceding definitions and 
hypotheses as long as the sets 

V{x) ■.= {{f{x,a),e{x,a)):aeA} 

are convex for all x G K^. This follows from the fact that if all the sets T>{x) 
are convex, then each relaxed control (3 £ A admits a measurable function a : 
[0, 00) — * A for which 

h''{y4s,P),Pis))ds^ f h{y4s,a),ais))ds Wt>0, h = f,£ 
Jo 
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We call T, A, f, and £ the target, control set, dynamics, and La- 

grangian for the problem Q , respectively. We let dS and 5^ denote the boundary 
and closure for any set S C M*^, respectively. 

The interpretation of our standing hypotheses is as follows. Condition (iJs) 
has the economic interpretation that all movement outside the target states is 
costly. Notice that {H^) is less stringent than requiring £{x, a) > for all x G 
and a € A, since it could be that points p for which mm{£{p, a) : a E A} ^ have 
the property that all inputs immediately bring p to points x where min{i{x, a) : a G 
A} > 0, which can give (H^) (cf. fjSlfor problems with this property). The condition 
(Hq) has the interpretation that trajectories that go further and further from the 
starting point without bound are unaffordable. In other words, trajectories that 
give finite total costs over [0, oo) must stay in some bounded set. As we show 
in below, {Hq) holds for a general class of shape-from-shading equations from 
image processing, as well as for problems with vanishing Lagrangians that are not 
tractable using the known results (cf. ^O] below). However, {Hq) does not follow 
from {Hi)-{H5) (cf. Remark l6 . 51 below) . Finally, we recall (cf. Chapter 3) that 
(iJs) guarantees that l@J admits a unique solution yx{-,l3) defined on [0, cxd) that 
satisfies 

sup \\ycc{t,u) -x\\< A'U for aU t e [0, 1/A4], (7) 

where '■= sup{||/(z, a)\\ : a d A, \ \z ~ x\ \ < 1} if this supremum is nonzero and 
Mx = 1 otherwise. 

The value function u of is defined by 

v(x)=inf|^^^''^'-(y,(s,/?),/3(s))ds:/3eAi.(/3)<oo| e [0, (8) 

(but see Remark 13.11 for extensions to problems with exit costs). This note will 
study viscosity solutions w of the HJBE 

sup {-fix, a) ■ Dw{x) ~£''{x, a)} =0, x^T (9) 

associated with the exit problem (j^J that satisfy the following side condition: 

(SCw) w is bounded- from-below, and w = on T 

We remark that the LHS in @ equals sup{— /(x,a) • Dw{x) — £{x,a) : a e A} 
(cf. [3]). When we say that a function w is bounded- from-belovif, we mean that 
there is a finite constant b so that w{x) > b for all x in the domain of w. In some 
of what follows, we use the notation 

Hb{x,p) sup{-/(a;, a) ■p-£{x,a)} 

aeB 
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for closed B C A. From {Hi)-{Hi), we know that Hb is continuous for all compact 
sets B C A. We sometimes write H{x,p) to mean Ha{x,p). We also set 

Bg{p) {a; e : \\x ~ p\\ < q} \/q > 0, p G M^. 

Letting C^{S) denote the set of all real- valued continuously differentiable functions 
on any open subset S of a Euclidean space, the definition of viscosity solutions can 
then be stated as follows: 

Definition 2.2 Assume C? C is open, S ^ Q, and F : M.^ x ^ R 
and w : S ^ R are continuous. We will say that w is a viscosity solution 

of F{x, Dw{x)) = {) on Q provided the following conditions hold: 

(Ci) If 7 e C^{Q) and Xo ^ Q are such that Xo is a local minimizer of it; — 7, then 
F{xo, Dj{xo)) > 0. 

(C2) If A G C^{G) and xi € G are such that xi is a local maximizer of w — A, then 
F{xi, DX{xi)) < 0. 

We also use the following equivalent definition of viscosity solutions based on the 
superdifferentials D^w{x) and subdifFerentials D^w{x) of w. Let Q, S, F, 
and w be as in Definition 12.21 and define 

DM^) := G : lim sup ^^^M^^^ilqili^^ < ol 

I Q3v^^ \\x-y\\ ) 

D-w{x) := L G M^: liminf "^^^ ' "[^^ ' ^ ' " > ol 
I g^v^x \\x-y\\ J 

One checks (cf. [3) that conditions (Ci) and (C2) are equivalent to 

(CO F{x,p) > for &\\xeg and p G 

{C'2) F{x,p) < for allx eg and p G L»+w(a;) 

respectively. Therefore, we equivalently define viscosity solutions by saying that w 
is a viscosity solution of F{x, Dw{x)) =QonQ provided conditions {C'i)-[C2) hold. 
Our results can also be extended to the case of discontinuous viscosity solutions 
(cf. i)7. II below for the definitions and extensions). 

3 Statement of Main Result and Remarks 

Our main result will be the following: 
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Theorem 1 Assume {Hi)-{Hq). If w : M.^ — > R zs a continuous function that is 
a viscosity solution of the HJBE ^ on M.^ \ T , and if w satisfies (SCw), then 
w = V. 



Remark 3.1 Under the standing hypotheses {Hi)-{Hq), if the value function v 
is finite and continuous on R^, then v itself is a viscosity solution of the HJBE 
(O on R^ \ T (cf. 3 ). Since v satisfies (SCy), Theorem ^ then characterizes v as 
the unique viscosity solution of the HJBE @ on M.^ \ T in the class of continuous 
functions w : M.^ — * R that satisfy (SCw)- The assumption that the control set A 
is compact can be relaxed in various ways (cf. ^S] below). Also, the statement of 
the theorem remains true, with minor changes in the proof, if we replace v with 

Vgix) = inf 1^ '''^^\-{y^{s,f3),f3{s))ds + g{y^{t^{p),p)) : /3 e AMP) < ooj 

for any continuous bounded-from-below final cost function g : M.^ R, except 
that the boundary condition in (SC^) that w = on T is replaced hy w = g on 
T . For extensions of Theorem^to discontinuous and local viscosity solutions with 
possibly unbounded control sets, see ii M7l8l I 



Remark 3.2 Theorem ^ applies to problems that are not tractable by means 
of the standard results from 3j or using |17[ 1181 1191 120L I22j . For example, the 
undiscounted exit time problem results of 127| require 

Ve > 0, 3Ce > s.t. e{x, a) >Ce 'iae A k ^ B{T, e), (10) 

where dist(a;,r) := inf{||x-6|| : b e T} and B{T,e) := {p e R^ : dist(p,r) < e}, 
i.e., uniform positive lower bounds for £, outside neighborhoods of T. In particular, 
P0() does not allow inf^ a) to vanish at any point outside T, nor does it allow 
control values a for which £{x,a) — » as ||a;|| — > oo. Moreover, as we saw in 
Example 1 1 . 1 1 ab ove . this condition cannot be dropped. The examples we consider 
in this paper do not in general satisfy H1U|) (cf. below). The results of |17[ I19j 
apply to exit time problems violating 11U|) and give conditions guaranteeing that 
V is the unique viscosity solution of the HJBE in a certain class of functions that 
are either proper (where properness of a function w means that w{x) — > +oo as 
\\x\\ — > oo) or that satisfy a suitable generalized properness notion. The results 
of |17l I19| require the positivity condition (H^), but they do not require {Hq). In 
, uniqueness results are given for problems that violate (|10ll but that do satisfy 




r{y^{s,(3),(3{s))ds <^ lim y,(s,/3) eT . (11) 



As we will show in [JHl below. Theorem ^ applies to physical problems from optics 
and image processing and to problems violating both (|1U|I and (|ll|l . including 
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variants of the Fuller Problem (cf. |17[ I19p . We remark that while the results of 
^lE] apply to cases where ((1(11) and ((TT|l both fail, the conclusions of those results 
are that if the value function is proper, then it is the unique proper solution of 
the HJBE satisfying appropriate side conditions. Since we do not need to assume 
properness in Theorem ^ our results can be viewed as an improvement of the 
results of J7] and ^3 for cases where the extra affordability condition (iJg) is 
also satisfied. Notice too that {Hq) can be expressed as 

/■oo 

/ i'' {yx{s, (3), (3{s)) ds <oo ^ snY>\\yx{s,P)\\<oo, (12) 

Jo s 

which is of course less restrictive than for problems with bounded targets (cf. 
iO below). I 



4 Main Lemmas 

Under our standing hypotheses {Hi)-{He), one proves (cf. 3 ) that the value func- 
tion u is a viscosity solution of the HJBE Q on \ T when v is finite and 
continuous. The proof follows easily from the fact that v satisfies the Dynamic 
Programming Principle, which asserts that 

vix) = mf^l^ r(2/,(s, a), a(s))ds + w(y,(t,Q))| V x £ (13) 

for all t G [0, infc tj;(a)[. Our uniqueness characterizations are based on the fol- 
lowing representation lemmas that say that viscosity solutions of the HJBE © on 
\T satisfy analogs of H13|l . The proofs of these lemmas are based on uniqueness 
characterizations for finite horizon control (cf. Chapter 3 of P]). 

Lemma 4.1 Assume {Hi)-{H4) are satisfied and u e C{E) is a viscosity solution 
of H{x, Du{x)) = on E, where E C is bounded and open. If we set Tq{(3) = 
mf{t > : yq{t,l3) £ dE} for each f3 e A and q e E, then, for all jS e A and 
q d E, we have 

uiq) < I t{yq{s,P),P{s))ds + u{yq{5,P)) (14) 
for < (5 < Tq{P). 

Lemma 4.2 Assume that the standing hypotheses (i?i)-(i?4) hold and that w G 
C{B) is a viscosity solution of the HJBE H{x^ Dw{x)) = on B, where B is open 
and bounded. Set 

Ts{p) := inf{ t : dist(yp(t, a), dB) < S, a e A} 
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for each p E B and 5 > 0. Then for any p E B and any S g]0, dist(p, 9B)/2], we 
have ^ 

w{p) > ji^f^jy £''{yp{s,a),a{s))ds + w{yp{t,a))^ (15) 
for Bl\te]0,Tsip)[. 

Notice for future use that we can also put 6 — Tq{f3) in H14|l when Tq{(3) < cx). We 
also need the foUowing consequence of the Bellman- GronwaU Inequality and the 
sequential compactness of A (cf . ) : 

Lemma 4.3 Let be a compact metric space, let {a„} be a sequence in A, and 
let c > 0. Assume / : xA^ satisfies (H^). Then there exists a subsequence 
of {an} (which we do not relabel) and an a E A such that the following conditions 
hold: 

1. ck„ —> a weak-* on [0, c]. 

2. If Xn X in M.^ , then 2/x„(', ctn) — ^ ?/x(', a) uniformly on [0, c]. 

Finally, we need the following variant of Barbalat's Lemma shown in Recall 
(cf. [221) that a continuous function g : M. ^ [0, oo) is said to be of class A4IC 
provided that g{0) = and that g is even and strictly increasing on [0, oo). For 
example, x ^ is of class MK. for all constants g > 1. Also, if G is any function 
of Sontag's Class /C (cf. [E]), then g{s) := G{\s\) is of class MIC. From [221, we 
recall the following: 

Lemma 4.4 Let g be a function of class AiK., (j) : [0, oo) — »• K be differentiable, 4>' 
be Lipschitz continuous, and L g{(l){s)) ds < oo. Then lim (/)(s) = lim (f>'{s) = 0. 

s — *oo s — >oo 

5 Proof of Main Result 

The proof that w < v pointwise is a repeated application of Lemma 14.11 that we 
leave to the reader (cf. 17 for details). It remains to show that w > v. We omit the 
superscripts r to simplify notation in some of what follows. The proof that w > v 
is similar in spirit to an argument from jl7[ I19j but with a weak-* argument and 
a localization based on {Hq) replacing the 'strong controllability' and properness 
conditions used in ^7j. Fix x G \ a constant k > w{x), and an integer J 
for which x G Bj{0). Set 

S^^{xeR^ : w{x) < k}, 
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which is open by the hypothesis that w is continuous. Set S = S^C] Bj{0), which 
is bounded and open. For each p 6 and (3 € A, set 



Fix 
Set 



Tj,{(3) inf{t > : yp{t, /?) e d{S \ T)}. 
e g]0, k — w{x)[. 



I{x,t,a) := / £{yx{s,a),a{s))ds + w{yx{t,a)) 

JQ 



wherever the RHS is defined. We also set 

Ts{p) = inf {t>0: dist {yp{t, a), d{S \T)) <5,aE A} 

for all p G and S > 0, and we define xi :— x, ti := Ti{xi) when Ti{xi) < +oo, 
and Ti:=10 when Ti(xi) = +oo. We can then use (|15|) of Lemma [4.21 to get an 
ai d A such that 

w{xi) > I{xi, Ti, ai) — e/4. 

(We will always assume that 6 of that lemma can be taken to be 1. Otherwise, 
replace Ti/k{xk) in what follows with Ts^{xk) for an appropriate sequence Sk I 0.) 
Note that y^i (ti, ai) G 5 \ T. By induction, we define 



Xk 



yxk-i{Tk^iiCtk-i) e S\T for fc = 2,3, . 



where 



(16) 



Tk 



Ti/k{xk) if Tiik{xk) < +00 
lO*"' otherwise 



Since Xk & S \ T, we can use ((T3|) to get an ak G A such that 

u;(a;fe) > I{xk,Tk,ak) ~ 2'^''+^^ for aU A: G N. (17) 
. . + Tfc, a J = limsupj, ak, and, for an arbitrary 



We also set ao — 0, ak ti + . . . 

ae A, 

ai{s) 



aj(s) 



if < s < (Ti, 

if (Ti < S < (72, 



a2(s - (Ti) 
a/c(s - cTfe-i) if cTfe-i < s < (Tfc, 
a if a,j < s, 



with the last line used if aj < +oo. (We use the subscript J to indicate the choice 
of radius in JBj(O).) From the definitions of Xk and aj, we know that 



yx{s,aj) = yx^is - ak-i,ak) e S\T when s < aj 



(18) 
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and 



i{yxk{s,ak),ak{s)) ds = / £{y3;{s,aj),aj{s)) ds > ior all k. (19) 

Jck^i 



Reapplying ((rzji . we therefore get 

w{x) > / £{yx{s,aj),aj{s)) ds + w{x2) - e/4: 



> I £{yx{s,aj),aj{s)) ds + w{x3) - e 1^ + ^ 
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> . . . 

> I{x, ak,aj) - I (l - ^) Vfc G N. (20) 

By (jl6|l and the boundedness of 5, we can find xj € S and a subsequence (which 
we will not relabel) for which x„ xj. (We later show that xj G d[Bj{0)] U T.) 
We claim that 

fj := inf{Tg,(Q;) : a € A} < limsupTfe. (21) 

fe 

To see why H21(l holds, first let S £ (0, oo) be given. Assume first that fj < oo. 
Suppose that for fc as large as desired we had Tk < fj — S. Passing to a subsequence, 
we can assume that Tfe ^ z € [0, fj — 5]. There would then exist a sequence f fc — > z 
and a control u G A such that 

dist(ys,(z,u),a(5\T)) ^dist(y,,(ffc,Ufe),9(5\r)) < 1/fc ^ 0, 

where we used the definition of the Tfc's and u is a weak-* limit of the u^'s on 
[0,fj — 6] (cf. Lemma [4 .31) . Since z < fj, this contradicts the definition of fj. If 
on the other hand we had fj = oo, then we arrive at the same contradiction by 
replacing fj — S with an arbitrary finite positive number in the previous argument. 
This establishes the claim H21|l . 

Using H21|l and passing to a further subsequence without relabeling, we can 
fix a constant I e [0, +oo] so that 

I > fj and Tfc t I- 

Moreover, the estimate Q for Lipschitz dynamics easily gives fj — iS xj £ 
d{S\T) (cf. Id for details). 

We now use a variant of an argument from ^3 to show that xj G d{S \ T). 
This argument, which is a consequence of the assumption (H^), is as follows. 
Suppose that xj ^ d{S \T), so I > fj > Q. Let M G (0, 1), and let a G ^ be a 
weak-* limit of a subsequence of the a^'s in A on [0, M], which we assume to be 
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the sequence itself for brevity (cf. Lemma [4. We conclude from 12U|) that 
^ t{yx{s,aj),aj{s)) ds 

Tkf\M I'M 

£{y^^{s,ak),ak{s)) ds / £{ys j{s,a),a{s)) ds. (22) 

Jo 

The left arrow is by the divergence test appHed to the integrals in (|20() , since w is 
bounded below and £ is nonnegative. The right arrow is justified by the argument 
of 

If we had J^'' £^{yxj{s,a),a{s)) ds > 0, then £^{yxj{s,a),a{s)) ds > for 
some G e (0, fj). Since / > fj, we would reach a contradiction by putting M — G 
in 1211). It follows that JJ''^ £'^{yxj{s, a),a{s)) ds = 0. Since we were assuming that 
xj ^ d{S \ T), we have fj > and xj ^ T, so this contradicts (iJs). Therefore, it 
must have been the case that xj e d{S \ T), as needed. Since 

d{S\T)Cd{S^)UTUd{Bj{0)), (23) 

we have the following cases to consider: 

Case 1: If xj £ 9(5k), then the continuity of w gives w{xj) = k. Using (|20|l . the 
nonnegativity of £, and the fact that e < k ~ w{x), we conclude that 

w{x) > w{xk) — e ^ w{xj) e > k — (k — w{x)) — w{x), 

which is a contradiction. Therefore, xj ^ 9(iSk). 

Case 2: If xj S T, then it follows from the controllability hypothesis STCT, the 
continuity of w, (SCw), and the estimate that there exist p £ N, f > 0, and 
(3 e A that are such that 

w{xp)>-elA, t :=t,^(/3) < TO, and / /3), /3(s)) < e/4. (24) 

Combining H20f) and H24|l now gives 

w{x) > I £{yx{s,a),a{s)) ds — e > v{x) — e, 
Jo 

where a is the concatenation of aj[[0, (Tp_i] followed by /3, and t^, := txia) < oo. 
This establishes that w{x) > v{x), by the arbitrariness of e. 

Case 3: Since Case 1 cannot occur, and since Case 2 gives the desired conclusion 
w{x) > v{x), it follows from (|23|l that we can assume that xj G d [Bj{0)] in what 
follows. 
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We may assume ctj < oo. (Otherwise, in what fohows, replace xj with one 
of the XkS for which \\xk\\ > J — and replace ctj with the corresponding 
fj/c-i- This is possible since Xfc ^ xj G i9[Bj(0)].) Notice that w{xj^ < k and 
xj = yx{^j, aj). Now repeat this procedure but with the initial value x replaced 
by xj, S replaced by iS^ n i?j+i(0), and e replaced by any positive number ei < 
e/2 A [k — 'w{xj)\ to get a trajectory for an input aj+i starting at xj which wlog 
reaches i9(i?j+i(0)) at time a,j+i < oo. If we now concatenate this result with 

<5j) [[0, cTj], then we get a trajectory that coincides with yx{-,aj) on [0,(Tj] 
and reaches d[Bj+i{0)] in finite time aj + ctj+i. 

This process can be repeated, with s replaced by any positive number Sq < 
A [k — w{xj+q-i)] and the starting point x replaced by xj+q-i in the qth 
iteration of this process. We can assume (Tj+g < oo and that all the points 
xj+q = ys:j^^_-^{aj+q,aj+q) obtained lie in d [Bj+q{0)] for all q, by the preceding 
argument. Set 



= cr.7 + (T.7+1 



aq and s — lim sup aq 



Fix b E A. We can then set 



aj+i{s - aj) 



a{s) 



:= < 



if < s < crj, 

if CTj < S < CTj+i, 



aj+qis - (yj+q-l) if CTj+q-l < S < (Tj+g, 



if s < 



to define an input a lE A. A passage to the limit as A: — > oo in H2U|) and a summation 
then gives 

w{x)> / £{yoois,a),a{s))ds + w{xq) - 2e for N3q>J. (25) 
Jo 

If s is finite, then we get 

d[Bj+q+i{0)] 3 ysj+^{aj+q+i,aj+q+i) = y.x{Bj+q+i,a) yx{s,a) as g ^ oo 

which is impossible. Using the fact that w is bounded-from-below, a passage to 
the limit as g — > oo in H25() therefore gives 



Since 



£{yx{s, ce), a{s)) ds < w{x) + constant < oo 



yxiaj+q+i,a) = yxj^^{aj+q+i,aj+q+i) G d[Bj+q+i{0)] 



(26) 
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for q = 1, 2, . . ., we also have 

limsup ||y^(s,d)|| = oo. (27) 

S — >C30 

But l|26|l - l|27(l stand in contradictfon with (Hq). Consequently, it must be the case 
that xj^q G T for large enough q. By the argument above, this gives the desired 
inequahty ■w{x) > v{x) and completes the proof. 

6 Three Applications 

This section shows how Theorem ^ applies to exit time HJBE's that are not 
tractable by means of the well-known methods, including cases where the methods 
of [T71 UHl Cni 1201 E2 cannot be applied. We also show how Theorem [J extends 
results from |19[I29| on degenerate eikonal and shape- from-shading equations from 
optics and image processing. 

6.1 Vanishing Lagrangians 

Theorem n can be used to give uniqueness characterizations for HJBE's that are 
not tractable using [T7| CHI d EOl E2 or [31 [T3| . For example, fix A: > 0, take 
N — 2, and use the exit time data 

T={{k,k)}, A=[-l,+l] 

f{x,y,a)^{y-m{x,y),a), £{x,y, a) ^ x^ + k{l ^ \a\)^ , 

where $ : — > [0,1] is any function that is 1 on Bk/4{{k, k)) and on 
— Bi.^2{{k,k)). The physical interpretation of this data is that $ guarantees 
STCT (cf. below), and the structure of £ penalizes inputs that are not bang- 
bang. This is a generalization of the Fuller Problem exit time problem data (cf. 
[miniliniEllEZI), which is the case where A: = in Recall (cf. 24 ) that the 
Fuller Problem admits a cost-minimizing control f3z for each initial state z gM."^, 
which is defined as follows. Set 

C := {{xuX2) ■■ \xi\ = Cxj, xiX2 < 0} C M^ 

set C"*" = {(2:1, 2:2) e C ■ ±2^1 > 0}, and let and denote the regions lying 
above and below C respectively, where C > is the constant root specified in pi) . 
Define the feedback k : M."^ ^ by k^q) = -1 if g G A" U C" , k{q) = 1 

if g G U C^, and fc(0, 0) — 0, and let jz be the closed-loop trajectory for the 
feedback k starting at z. We then take l3z{t) = +1 if jz{t) G A+, l3z{t) = — 1 if 
7z(i) G A", and Pz{t) = if ^zit) = (0,0). Let Vk denote the value function ijSJ) 
for the exit time problem with data H28|) . 
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As shown in |52] (see also HHj), the value function v = Vo for the Fuller 
Problem is the unique bounded-from-below viscosity solution of the corresponding 
HJBE on \ {0} in the class of all continuous functions w : ^ R that are 
null at (0,0). This result uses the fact that the Fuller Problem satisfies On 
the other hand, for fc > 0, the exit time data H28|) violate both (|1U|) and For 
example, (tTUIl is violated since £(0,p, 1) = 0, even though (0,0) ^ T. Therefore, 
the data is not tractable using pilTll?/]. 

To see why (lllf) fails for fc > 0, let y'^(-, a) denote the trajectory for the data 
H28|l . the control a, and the initial position q. For n e N and (3z as defined above, let 
p{n) := (l/(2n2),l/n) = y°_„,(l/n,a = 1) and t„ := inf{t > : y° „,(i, /3p(„)) = 
(0,0)}. Using [37|, we have M := sup{t„ : n e N} < oo. Let /3 denote the 
concatenation of /3p(i) [[0, ti] followed by a = 1 [[0,1/2] followed by /3p(2) [[0, 
followed by a = 1[[0, 1/3] followed by /3p(3) [[O,^] followed by a = 1[[0, 1/4] and 
so on. Since the norm of the first coordinate of y°(„)(-, /3p(n)) is always below 
(cf. PH), Wo(p(n)) < M/n^ for aU n. For n>2, set 



-1 



SO p(n) — y°(jj(i„, /?). Since H28(l agrees with the Fuller Problem data for {x,y) in 
some neighborhood of and \a\ = 1, each fc > admits an n{k) e N such that 
y;(„(fc))(s>7n(fc)) = yp(„(fc))(«'T«w) fo"^ all s > 0, so 



^iyp(n{k))is,"/nik}),lnik){s))ds = ^ 

n— n(/c) _ 



l/(n+l) 



< [M/n^ + 1/(20 n^)] < oo, 

n—n{k) 

even though yp(„(fc))(s,7n(fc)) (0, 0) ^ T as s ^ +oo. 

One checks that {Hi)-{Hq) hold for for all k > 0. For example, (H^) holds 
since the dynamics in l|28|l agrees with the Fuller dynamics in a neighborhood of 
the 2/-axis and the Lagrangian £ assigns a positive cost to staying at (0,0) when 
A; > and the Fuller Problem satisfies {H5). The fact that STCT holds for (gHl 
follows since f{x,y,a) = (y — k,a) near (fc, fc) and the Fuller Problem satisfies 
^TCj (0,0)} (cf. 5ni)i along with a change of coordinates. Finally, condition (Hq) 
holds by Lemma [4 . 41 with g{x) := x^. This application of Lemma f4. 41 is based on 
the fact that $' has compact support, which guarantees that the second derivative 
of the first component of y^{s, (3) is globally bounded. We conclude as follows: 

Corollary 6.1 Let fc > be constant, and choose the exit time problem data 
H28|) . If u) : ^ R is a continuous function that is a bounded-from-below viscosity 
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solution of the corresponding HJBE 

i-y + k<^>{x, y)] {Dw{x, y))^ + \iDw{x, y))2 1 - = 
on \ T that is nuU at T, then w = Vk- 

Taking = in Corollary 16. II gives the uniqueness characterization for the Fuller 
Problem HJBE asserted in 22 . The novelty of Corollary |0] is that it applies to 
problems violating both the usual positivity condition (fTU]) and the asymptotics 
condition from [21], and that it establishes uniqueness of solutions of the 
HJBE in a class of functions that includes functions that are not proper. 



Remark 6.2 Using the fact that x x"^ is convex, one shows that Vo is convex on 
M."^ and therefore continuous. Moreover, using Soravia's Backward Dynamic Pro- 
gramming Principle (cf. pil27p. one can show that {x, y) i— > w{x, y) :— — Wo(— a;, y) 
is also a viscosity solution of the Fuller Problem HJBE on \ {0} vanishing 
at the origin. The argument is based on the facts that Vo is a bilateral viscosity 
solution of the HJBE and that each p e is an optimal point (cf. 3 for the 
definitions) and the fact that (^1,^2) G D^w{x,y) (pi,— P2) £ D^Vo{—x,y) 
and that (pi,P2) G D~w{x,y) ^ (pi,— P2) G D'^Vo{^x,y). It follows that Vo is 
the unique continuous bounded-from-below viscosity solution of the corresponding 
HJBE on R2 \ {0} that vanishes at the origin and that the boundedness from below 
hypothesis of CoroUarv 16 . II cannot be removed. I 



Remark 6.3 Corollarv l6.1l can be generalized. For example, the corollary remains 
true if the Lagrangian £ in is replaced by y, a) = g{x) + — \a\f for any 
g of class M.1C, e.g., g{x) — \x\'^ for any q > 0. The proof goes through without 
changes if the data are modified in this way. Also, the target T = {(fc, fc)} can be 
replaced by {(fc, m)} for any fc 7^ and any m G M if $ is chosen to be 1 near (fc, m) 
and zero in some open set containing the y-axis. Moreover, using the methods of 
fJZI below, the above corollary can be extended to cover local and discontinuous 
viscosity solutions. I 



6.2 Degenerate Eikonal Equations 

This subsection shows how Theorem^applies to the HJBE's for a class of exit time 
problems from geometric optics. The problems have the dynamics f{x,y,a,b) = 
(a, b) e Bi{0) C R2 and the Lagrangians 



iix,y,a,b) = l + ^\\ix,y)\\ 



(29) 



where p > is a constant that will be further specify below. (The argument 
we are about to give also applies if we instead take the Lagrangian (1 + \/|x|)^ 
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or (1 + \/\y\)^, or if the state space and compact control set are in R*^ for M 
arbitrary.) 

We choose any nonempty closed target T C M^, and we let Ve,p denote the 
value function for the exit time problem we have defined for each p > 0. The 
corresponding HJBE is 

\\Dvix,y)\\ = ^ ^ (30) 



which is the eikonal equation of geometric optics for the propogation of light in a 
medium with speed 

c{x,y) = [^ + V\\ix,y)\\Y ■ 

Viscosity solutions of eikonal equations have been studied extensively (cf. [3] , which 
covers cases where the speed of the medium is bounded and also uniqueness ques- 
tions for eikonal equation solutions on bounded sets, and PHj). However, (|3()|l is 
not covered by these results since c is unbounded and T may be unbounded. It 
is easy to check that for < p < 2, the exit time problems for these data satisfy 
{Hi)-{Hq). Indeed, if g G and if (j) is any trajectory for / starting at q, then we 
can find a iiT > so that, for each L > K, we have 



ds 



> 



ds 



i+v4Mr 



> 



K 



ds 

W2 



as L ^ OO, SO (Hq) is satisfied vacuously. We conclude as follows: 

Corollary 6.4 Let p € [0, 2] and T C closed and nonempty. If w : R^ ^ R 

is a continuous function that is a bounded-from-below viscosity solution of (|30|l 
on R^ \ T that is null on T, then w = v^ ^. 

Remark 6.5 It was not necessary to assume that the target T is bounded. If 
p > 2 in (|29ll . then Theorem may not apply, since {Hq) could fail. For ex- 
ample, if p = 4, and T = {{x,0) e R^ : a; < -1} and /? = (1,0), then ^ 
gives £(j/(o,o)(s, /3), /3(s)) ds < oo, even though the trajectory does not re- 
main bounded. Moreover, the standard uniqueness characterizations for exit time 
HJBE's (e.g.. Corollary IV. 4. 3 of 3 ) would not apply, since (|10|) is not satisfied. 
However, using [SOj, one can show that the statement of CoroUarv lB . 41 remains true 
even without the restriction p e [0, 2]. This is done by rewriting the HJBE H30|l as 



1 + V\\ix,y)\\ \\Dv{x,y)\\-1^0 



(31) 



and then viewing H31|l as the HJBE for the exit time problem with the non- 
Lipschitz dynamics 



■1 P 



f{x,y,a,b)^ 1 + y/\\{x,y)\\ (a, 6) 
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(with (a, 6) G Bi{0) as before) and the Lagrangian i = 1. The dynamics / is then 
approximated by locally Lipschitz dynamics, and then Theorem IV. 4. 4 of 3 is 
applied. For details, see §6.1 of I^Ol- ■ 

6.3 Shape-From-Shading Equations 

Our results also apply to equations of the form 

I{x)'i'{Du{x)) - b{x) ■ Du{x) - h^(x) = 

for / nonnegative and \1/ : M any convex function with '5(0) = 0. This 

equation is studied in [211 ■ Taking the Legendre transform 'J* of 'J, which is 
nonnegative, we can rewrite this equation as 

max {-{h{x) - I{x)a) ■ Du{x) - [h^ix) + /(x)**(a)] } = 0. 

a€:domain(\I'*) 

A particular case of this equation (cf. [221) 

I{x)[l + \\Du{x)\\^f^ -l = Q, xencR^ (32) 
for open sets f2, which in fact can be written as 

max^ |/(a;)a • Du{x) - 1 - I{x) (l - | \a\ \^) ^'"^ } = 0. (33) 

The equation l|33|l arises in shape-from-shading models in image processing, where 
/(x) G [0, 1) is the intensity of hght reflected by an object (cf. [SOD- The objective 
in image processing is to reconstruct the unknown function m, representing the 
height of the surface on some subset of the plane, from the brightness of a single 
two-dimensional image of the surface. For the case of a Lambertian surface that is 
not self-shadowing and that is illuminated by a single distant vertical light source, 
the height u is a viscosity solution of l|33|) . 

Now pick any closed nonempty target T C \ {0} and := \ T, and 
choose the intensity function 

:^ (34) 

Then (|33|l is an HJBE for an exit time problem with the dynamics 

](x,u) := -I(x)u, (35) 



the control set A — -Bi(O) C R^, and the Lagrangian 



£(x,z.) = l- (36) 
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As explained in Remark 13.21 for general T C \ {0}, (. violates the positivity 
condition 1(1^ (since t{x,Q) ^ as ||a;|| oo), so the well-known results (e.g., 
those of O) cannot be used to get uniqueness characterizations for solutions of 
H32|) . On the other hand, using the fact that 

||y,(s,/3)|| < \\q\\ + S 

for all /3 e ^, s > 0, and q e M^, one can easily check that {Hi)-{Hq) hold. The 
argument is similar to the validation of (Hq) in i)6.2l Therefore, we conclude from 
Theoremnthat if it; : — > R is a continuous function that is a viscosity solution of 
H33|) on \T that satisfies (SCw), then w coincides with the shape-from-shading 
value function. Local uniqueness characterizations and results for discontinuous 
viscosity solutions for the shape-from-shading equation can also be given using 
the results in below. 

Remark 6.6 As in the case of eikonal equations, it was not necessary to assume 
that the target T was bounded. It is worth remarking that if we replace the light 
intensity I{x) with 

3e2|kll 

in the previous example and keep the example the same otherwise, then Theorem 
d would no longer apply, since condition {Hq) may not be satisfied. However, for 
such cases, we can still apply to get uniqueness of proper solutions of the 
corresponding HJBE. For example, take T = {(0,r) : r < —1} and the control 

/3(t) = (0,-l/(<+l)), 

and let t i— > x{t) = {xi{t), X2{t)) denote the trajectory of f{p, u) := —I{p)u for the 
initial position p(0) ~ (0, 1) and the control u ~ (3{t). For all i > 0, we then have 
xi{t) = 0, 

1 , . 3 /■* 1 , 3 



X2{t) = 1+ I{x{s))—-ds > 1 + - ——ds = l + -ln(t + l), 

S + 1 4: Jq s + 1 4 







so x(t) is not bounded. However, 



2||a;(t)ll||^(i)||2 ^ _J_e[2+2/o ^>(''))tFtH 



< 1 e2g21n(t+l) 



< e^. (37) 

Therefore, if i{p,u) = 1 — I{p)[l ~ denotes the corresponding La- 

grangian, then since we have 



£{p,u) < 1 - /(p)[i - \\u\\^] \fp e R', ue Bi(0), 



20 



l|57|l gives 







3e2||a;(s)| 

dt 



3g2+3/21n(t+l) 



/•oo 



even though 1 1—^ x{t) is not bounded, which shows (Hq) is not satisfied. Moreover, 
the standard uniqueness characterizations for exit time HJBE's (cf. 013) would 
again not apply, since the Lagrangian £ is not uniformly bounded below by posi- 
tive constants. However, since (H^) holds, one can use ^7] to show that for any 
nonempty closed target T C \ {0}, any proper continuous viscosity solution of 
the corresponding HJBE 



sup \i{x)a ■ Du{x) - [l - I{x) (l - ||a|H^^^] } 

||a||<l L -I J 



= 



on \ T that is null on T must in fact be identically equal to the shape-from- 
shading exit time value function v^^^ for the target T, the dynamics /, and the 
Lagrangian £. I 

Remark 6.7 Notice that it was not necessary to assume that the domain set 
for H32|l was bounded. It is worth pointing out that one cannot in general expect 
uniqueness of solutions for the shape-from-shading HJBE for cases where / is 
allowed to take the value 1, since the surface u and —u could both be viscosity 
solutions of l|32|l . For example, take the light intensity I{x) = (1 + 4| |a;| 
T ^R^\ Si(0), and the surface u{x) = 1 - || a; IP on -Bi(O) and zero elsewhere. 
Clearly, u and —u are both solutions of (|32|l . However, {H5)-{Hq) are not satisfied, 
since the trajectory (j)(t) = gives zero integrated costs on [0, oo) without ever 
reaching the target, so this case is not covered by Theorem ^ For the analysis of 
cases where #{x : I{x) = !} = !, see and for bounded viscosity solutions of 
see EHI- ■ 



7 Discontinuous and Local HJBE Solutions 

This section gives variants of Theorem Q] for discontinuous and local HJBE solu- 
tions. We study discontinuous solutions using the envelopes approach from 

7.1 A Remark on Discontinuous Viscosity Solutions 

Under [Hi)-{Hq), the value function Vg could be discontinuous (cf. [S|, pp. 248- 
249). This suggests the question of how one can characterize v as the unique dis- 
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continuous solution of the HJBE on M.^\T that satisfies (SCy). By a discontinuous 
solution, we mean the following. For each locally bounded function w : S ^M. on 
a set S C , we define the following semicontinuous envelopes: 

Wi,{x) := limini w(jj) and linisup . 

We call w^, the lower envelope of w, and we call w* the upper envelope of w. 
For Q, S, and F satisfying the requirements of Definition 12.21 wc then say that 
a locally bounded function w : S' ^ R is a discontinuous subsolution (resp., 
supersolution) of F{x, Dw{x)) = on Q provided F{xo, Dj{xo)) < (resp., 
> 0) for each 7 G C^{G) and each local maximizer (resp., minimizer) of — 7 
(resp., lu* — 7) on Q.^ A (discontinuous viscosity) solution of F{x, Dw{x)) = 
on Q is then a function that is simultaneously a discontinuous subsolution and a 
discontinuous supersolution of F{x,Dw{x)) ={) onQ. Lemma |4. II remains true if 
u £ C{E) is replaced by any bounded discontinuous subsolution of the HJBE on 
E and u in H14|) is replaced by u* . Also, Lemma f4.2l remains true if w S C{B) is 
replaced by any bounded discontinuous supersolution of the HJBE on B and w in 
(|15|l is replaced by w*. Using these facts, one can prove the following generalization 
of Theorem n If {Hi)-{Hq) hold, if w : — *■ R is a discontinuous viscosity 
solution of the HJBE on R^ \ T that satisfies (S'C^), and if Wi, is continuous on 
R^, then w = v on R^. ^ The proof is almost identical to the proof of Theorem ^ 
but with w replaced by Wi, in the proof of the inequality w > v. For cases where v 
is continuous, this establishes that all solutions w of the HJBE (O on R^ \T that 
satisfy (SCw) and continuity of w;* agree with v, and therefore are continuous. 

7.2 Local Solutions of the HJBE 

This subsection shows how to extend Theorem ^ to get uniqueness of solutions of 
the HJBE on sets of the form \ T for open sets fi. We set 

n= {xeR^ : inf{t^(/3) : (3 e A} < 00} , 

so TZ is the set of points that can be brought to T in finite time using the dynamics 
/. Using [H2)-{Hz), one shows that TZ is open (cf. [H]). In many classical cases 

^In this context, 'discontinuous' means "not necessarily continuous". 

■^The continuity of ui* is used to ensure that the sets Sn in the proof of Theorem Q are open. 
The condition that Wf, is continuous of course holds automatically if w is continuous. However, 
{SCw) and continuity of u;* can even be satisfied by functions that are nowhere continuous. For 
example, if we take the indicator function ui = 1q : R — > {0, 1}, then id* = 0. This generalized 
version of Theorem remains true if the pointwise condition that «) = on T is replaced by the 
less restrictive requirement that there be a locally bounded function g : M.^ — > R for that 

S T, Wi,{x) > g-^ix) and w*{x) < g* {x) 

except that the conclusion that w = v is replaced by the following inequalities on M.^ (cf. Remark 
13. li : «)* > Vg^ and w* < Vg* . In case g £ C(T), this implies w = Vg on M.^ . 
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where ^ is bounded below by a positive constant, one has 



V is continuous on 7?., and hm v{x) — +00 \/xo G dTZ. 



(38) 



On the other hand, one easily finds examples where £ is not bounded below by 
a positive constant and the limit condition in (|38|l fails. Here is an elementary 
example where this occurs: 

Example 7.1 Take iV = 1, T = [1, +00), A = {+!}, /(x, a) = \x\a, and t{x, a) = 
\x\. In this case. 



This motivates the question of how one can characterize w as a unique viscos- 
ity solution of the HJBE on 7^\T for cases where TZ ^ and the extra condition 
H38|) holds. To address this question, we assume the following relaxed version 

{H'^) If X e 7^ \ r and /3 e A then /J r(y^(s, /?(s)) ds > for all 

t e (0,00). 

of (-ffs). We also fix an open set Q, C TZ containing T, and we consider viscosity 
solutions of the HJBE ^ on \ T that satisfy the localization 

{OSC^ ii) w is bounded-from-below on Q, w = on T, and lim w{x) = +00 



Noting that v satisfies iOSC^ -n) if (|38|l holds, we then have the following local 
version of Theorem ^ 

Theorem 2 Let {Hi)-{H4), {H'^), and {H^) hold. Let il C TZ be an open set 
containing T . Let w : Q ^ 'M. be a continuous function that is viscosity solution of 
the HJBE 0^ on n\T that satisfies (OSC^ a)- Then, w = v on fl. In particular, 
if V satisfies then v is the unique viscosity solution w of the HJBE on TZ\T 

in the class of all continuous functions w : TZ M. that satisfy [OSC^ t^). 

Remark 7.2 The proof of the inequality w <v ior Theorem|5]is exactly the proof 
of that inequality in 19 . The proof is slightly more complicated than the proof 
that w < V for Theorem ^ since one must consider trajectories that reach T in 
finite time but that exit fl before the first time they ever reach T. The proof of 
the reverse inequality closely follows the proof of Theorem ^ except that instead 
of setting 5 = 5^11 Bj{0), we set 5 = iSk n Bj{0) n il. We rule out cases where 
xj G dfl using the limit condition in (OSC^ n)- Theorem|5|can also be generalized 
to the case of discontinuous viscosity solutions using the method of ii7.1l 




.ln(l/2) 



xe* dt = 1 — X ^ 1 as x J, 0, 



even though e dTZ. 



Vxo e on. 
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8 Problems with Unbounded Control Sets 



We close by giving two variants of Theorem ^ that can be apphed for cases where 
the control set A C R*^ is closed but possibly unbounded. In the first variant, 
we impose regularity conditions on the data that penalize the use of control set 
values of large norm. In the second variant, we replace the possibly unbounded 
control set A with a suitable compact set of vector field valued controls. Recall the 
definition © of that applies to possibly noncompact control sets. 

8.1 Penalization Method 

For simplicity, let us assume that all the sets 

V{x) ■.^{{f{x,a),£{x,a)):aeA} 

are convex. As explained in fJ21 the set of inputs a Q A can then be taken to be the 
measurable functions valued in A (by the Filippov Selection Theorem). We assume 
that {H2)-{He) are satisfied, where e ^ C E*^ for M g N and A is closed but not 
necessarily compact. Following Eli we then add the following conditions on 

/ and £: 

{H-j) f is bounded on 5^(0) x A for each R> 0. 

{Hs) There is a modulus uj such that \£{x,u) — £{y,u)\ < llj{\\x — ?;||) for all 
x,y e and u <E A. 

(Hg) There exist constants £o > 0, Co > 0, P € (0, 1], <52 > 0, / > 0, and 
^1 > 1 such that the following conditions hold for all x,y G and 
a e A: 

(a) £{x,a) > £o\\a\\^' ~ Co 

(b) \£{x,a) - £{y,a)\ < £\\x - y\\0{l + \\a\\'^ + \\x\\'^ + \\y\\'^) 

(Recall that a modulus is a nondecreasing continuous function uj : [0, oo) 
[0, oo) for which uj{0) = 0.) As shown in j^. Lemmas 14.11 and 14.21 remain true if 
{H2)-{Hs) are assumed instead of the assumptions {Hi)-{Hq). These assumptions 
penalize the use of control set values of large norm. We then consider only viscosity 
solutions w of the HJBE on \ T for which the subdifferential sets D~w{x) are 
locally bounded, i.e., such that sup{||p|| : p S D~w{x),x G K} < oo for each 
compact set K C R^. As shown in Theorem 1.7.3 of 0, this is equivalent to 
considering only locally Lipschitz solutions of the HJBE on R^ \ T. In this case, 
the infimizations in the restriction of the HJBE to any Bj{0) can be taken over 
a corresponding compact set Cj C A, i.e., in the notation we introduced in 32 
Ha \[B,j{0) X Dj] = He J \[Bj{0) x Dj], where Dj is a bounded set large enough to 
contain {p € D^w{x) : x G Bj{0)} (cf. 01 E] for the proof). Then the arguments 
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in 321 on Bj{0) apply with the compact control set Cj replacing A, and then we 
iterate on J to get an input a : [0, oo) ^ A as before. We then invoke {Hq) to 
conclude as follows: 

Theorem 3 Assume hypotheses {H2)-{Hq), with A a closed set containing G 
R^^ . Let w : — > M 6e a locally Lipschitz function that is a viscosity solution of 
0) on \ T that satisfies (SCw)- Then w = v. 

8.2 Vector Field Valued Controls Method 

Another way to extend Theorem ^ to the case of noncompact control sets is as 
follows. As in the previous subsection, we assume the sets ^{x) are all convex. We 
give C(R^,R^ X M) the topology of compact convergence (cf. We continue 

to assume {H2)-{Hs) and that A C M*^ is closed and nonempty but possibly 
unbounded. We also add the following assumptions: 

(NCi) sup{e{0,u):ueA}<oo. 

iNC2) {(/(•, It), : u e A} C C(R^,R^ x M) is closed. 

These guarantee that the supremum in the definition of the HJBE is always finite. 
It follows from the Ascoli-Arzela Theorem that K {ku{-) ■= {f{-,u),£{-,u)) : 
u € A} is a compact subset of the metric space C(M^,R^ x R) (cf. [II ESI)- 
Define the projection mappings nj on K by 

We now apply the method of our proofs to the new exit time problem whose 
dynamics F, Lagrangian A, and set A of admissible controls are 

F{x, k) = (tti o k){x), K(x, k) = [1:2 ° k){x) k A -.^ {[0, 00) 9 i i--> kpi^t) : P e A} 

with the same target T. Notice that {{F {x , k) , A{x , k)) : k e K} is convex for 
each X £ R^. Let v denote the value function of this new problem. Since the 
trajectories of F with the controls A are exactly the trajectories of / with controls 
in A, it follows that v = v. Moreover, the new problem satisfies {Hi)-{Hq) (with 
K replacing A^ F replacing /, and A replacing I). Our proof of Theorem ^ then 
gives the following: 

Theorem A Let $ ^ A <Z M*^ he closed. Assume {H2)-{Hs) and (NCi)-{NC2). 
Let w : R"'^ M. be a continuous function that is a viscosity solution of 0) on 
-^N \^ q- ^^^^ satisfies {SCw). Then w = v. 
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We remark that if {H2)-{Hs) and {NCi)-{NC2) all hold with ^ ^ a closed subset 
of M^, and if 7^ = M.^ , then the value function ?; is a discontinuous viscosity 
solution of the HJBE on \ T (cf. jH])- If we also assume is continuous, 
then a generalization of Theorem ^ characterizes v as the unique discontinuous 
viscosity solution w of the HJBE in the class of functions w : — > R that satisfy 
(SCw) and continuity of w*. The generalization of Theorem 0| to discontinuous 
solutions follows from the argument of ^7.11 Also, the theorem extends to local 
HJBE solutions using the arguments of the previous section. 
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